Reinforcement Learning for a Visually-guided Autonomous Underwater Vehicle

Reinforcement learning uses a scalar reward signal and much interaction with the environment to form a policy of correct behavior. We have applied this technique to the problem of developing a controller for an autonomous underwater vehicle and have achieved reliable off-line development of stable controllers. Many important underwater tasks rely upon on visual observation of underwater features. We have devised a feature tracking method and a vehicle guidance scheme that are also based on visual observation of features. We have obtained results in reliably tracking features in underwater imagery, not for map building but to guide an underwater vehicle. Using visual servo control techniques, feature position can be used directly to guide motion.